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1. Introduction 


We begin with introducing some notations that will be used throughout the paper. As¬ 
sume that (^rt)55TQ is a sequence of arbitrary random variables taking values from a finite set 
of X = {1, 2, • • • ,b} and (U, J", P) the underlying probability space. For convenience, denote 
by the random vector of • • • ,^m+n) and Xm,n = {xm, • • • ,Xm+n), a realization of 
^rn,n- Suppose the joint distribution of is 

'Pi(Tn,n = Xm,n) = P{xm,- ■ ■ ,Xm+n) = P{xm,n), Xk € m < k < ITl + U. (1.1) 
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Let (an)^o be two sequences of nonnegative integers such that (j){n) converges 

to infinite as n —> oo. Let 


fa„,4>{n){^) ^(n) 


( 1 . 2 ) 


where log is the natural logarithm. fan,<p{n)i^) will be called generalized entropy density of 
^an,<p{n)- If On = 0 and <()(n) = n, /o„, 0 (ra)(w) will become the classical entropy density of 
^o,n defined as follows 


/o.«(w) =--logp(Co.n)- (1-3) 

n 

If i^n)^=o is a nonhomogeneous Markov chain taking values in finite state-space of X = 
{ 1 , 2 , • • • ,b} with the initial distribution 


(Mo(1), • • • ,Mo(&)), (1-4) 

and the transition matrices 


Pn = {Pn{i,j))bxb, n = I,2' 

where Pn{i,j) = P(Cn = j'lCn-i = i), then 


/a„,0(n)(w) = l0g/Xa„(Ca„) + Pk{^k-1, ^k) 

' k=a„ + l 


an,+(^(n 


where pan (^) is the distribution of ^a„ ■ 


(1.5) 


( 1 . 6 ) 


The convergence of /o,n(w) to a constant in a sense of £i convergence, convergence in 
probability or a.e. convergence, is called Shannon-McMillan-Breiman theorem or entropy 
ergodic theorem or asymptotic equipartition property (AEP) respectively in information 
theory. Shannon [10] first estabilished the entropy ergodic theorem for convergence in prob¬ 
ability for stationary ergodic information sources with finite alphabet. McMillan [9] and 
Breiman [3] obtained, for finite stationary ergodic information sources, the entropy ergodic 
theorem in £i and a.e. convergence, respectively. Chung [ 6 ] considered the case of countable 
alphabet. The entropy ergodic theorem for general stochastic processes can be found, for 
example, in Barron [2], Kieffer [ 8 ], or Algoet and Cover [1]. Yang [12] obtained entropy er¬ 
godic theorem for a class of nonhomogeneous Markov chains, Yang and Liu [13], the entropy 
ergodic theorem for a class of mth-order nonhomogeneous Markov chains, Zhong, Yang and 
Liang [14], entropy ergodic theorem for a class of asymptotic circular Markov chains. 

The second term of Eq. (1.6) is actually delayed sums of random variables, which was 
first introduced by Zygmund [15] who used it to prove a Tauberian theorem of Hardy. 
Since then, a lot of work has been done to investigate the properties of delayed sums. For 
example, by using the limiting behavior of delayed sums, Chow [4] found necessary and 
sufficient conditions for the Borel summability of i.i.d. random variables and simplihed the 
proofs of a number of well-known results such as the Hsu-Robbins-Spitzer-Katz theorem. 
Lai [11] studied the analogues of the law of the iterated logarithm for delayed sums of 
independent random variables. Recently, Gut and Stradtmiiller [7] studied the strong law 
of large numbers for delayed sums of random fields. 


Let (^n)^o Le a nonhomogeneous Markov chain with the transition matrices (1.5). Yang 
[12] showed that the classical entropy density /o,„(w) of this Markov chain converges a.e. 
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to the entropy rate of a Markov chain under the condition that lim„_>oo ^ ELi \pk{^,3)- 
Pihj)\ = 0, for all i,j € X, where P = ip(i, j))bxb is an irreducible transition matrix. In 
this paper, we will prove that the generalized entropy density /o„,0(n)(w) converges a.e. and 
Cl to this entropy rate under some mild conditions, which is called the generalized entropy 
ergodic theorem. The results of this paper generalize the results of those in [12]. 

To prove the main results, we first establish a strong limit theorem for the delayed sums 
of the functions of two variables for nonhomogeneous Markov chains, then we obtain the 
strong limit theorems of the frequencies of occurrence of states and the ordered couples of 
states in the segment Ca„, Ca„+0(n) for the Markov chains. At the end we present the 

main results. We also prove that /a„,0(n)(w) are uniformly integrable for arbitrary finite 
sequence of random variables. 

The approach used in this paper is different from the one used in some previous works([12],[13]), 
where the strong law of large numbers for martingale is applied. As /a„,0(n)(w) is the delayed 
sums of \ogpk{^k-i,ik), the strong law of large numbers for martingale cannot be applied. 

The essence of the technique used in this paper is first to construct a one parameter class of 
random variables with means of 1, then, using Borel-Cantelli lemma, to prove the existence 
of a.e. convergence of certain random variables. 

The rest of this paper is organized as follows. In section 2, we first establish some 
preliminary results that will be used to prove our main results, and present the main results 
of this paper and their proofs in section 3. 


2. Some lemmas 

Before proving the main results, we first begin with some lemmas. 


Lemma 1. Suppose {^n)'^=o is a nonhomogeneous Markov chain taking values from a finite 
state-space ofH. = {1, 2, • • • ,b} with the initial distribution (l.f) and the transition matrices 
(1.5). Suppose (an)[]Tg and ((/>(n))[]Tg are two seguences of nonnegative integers such that 
4>{n) tends to infinity as n ^ oo. Let {gnix,y))'i^^Q be a seguence of real functions defined 
on X X X. If for every £ > 0 

CXD 

^ exp[-£(/)(n)] < oo, (2.1) 

n—1 

and there exists a real number 0 < 7 < oo such that 


lim sup 

n 


an+rt>{n) 


c(j;uj) < 00 a.e., 


( 2 . 2 ) 


then, we have 


lim 

n 


1 

(fin) 


a„+0(n) 

E {9kif.k-i,fk)-E[gkifk-i,fk)\f,k-i]} = 0 a.e. 

k—an-\-l 


(2.3) 


Remark 1. Obviously, condition (2.1) in Lemma 1 can be easily satisfied. For example, 
let (j){n) = [n“](a > 0), where [•] is the usual greatest integer function, then (2.1) holds. If 
i9nix,y)))(Li are uniformly bounded, then Eq. (2.2) holds. 
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Remark 2. Since E[gk{^k-i,^kMk-i] = Yfj=i9k{^k-i,j)Pk{^k-i,j), Eq. (2.3) 
rewritten as 


lira 

n 


1 


a„+tj>{n) b 

X! {9ki^k-i,^k)-'^gk{^k-i,j)Pk{^k-i,j)} = 0 a.e. 

k—an-\-l j — 'i- 


Proof. Let s be a nonzero real number, define 




eMsEtltt^:l9kifk-i,fk)} 




, n = l,2,' 


and note that 

—^\^\-^an,<p{n) (^j |'Co,an+(/*(n) —l]] 

^^9an-\-(i>{ri) (Can+ — 1 ’Can + </>(Ti) ) 


=E 


=E 




ICo,a„+ 0 (n) —; 


^-^an.ipin) — ! (s, Cc) 1 . 

For any e > 0, by Markov inequality and Eq. (2.1), we have 

OO 

[(/)"^(n) logAa„_,^(„)(s,w) > e] 

n —1 

OO 

= ^P [Aa„_,^(„)(s,u;) > exp((/)(n)£)] 

n —1 

OO 

< ^ 1 • exp(—0(n)£) < OO. 

n —1 

By Borel-Cantelli lemma and arbitrariness of e, we have 

limsup-^logAa„, 0 („)(s,a;) < 0 a.e. 

n ) 


Note that 


(f{n) 


log Aa„,0(„)(s, w) 


1 


a„+cl>{n) 


(j){n) 


Y, {s5fe(a-i,a) -logE[e^9'=l«'=-’«'=)|5fc_i]}. 


k—an-\-l 

By Eqs. (2.7) and (2.8), we have 

a„+<j>(n) 


limsup^^ Y, |sfffe(Cfe-i.Cfc) - logE[e^®*’l^'“-i’^'“l|^fe_i]| < 0 


fc=an + l 


can be 

(2.4) 


(2.5) 

( 2 . 6 ) 

(2.7) 

( 2 . 8 ) 

(2.9) 



Letting 0 < s < 7, dividing both sides of Eq. (2.9) by s, we obtain 

a„+</>(n) , , 

limsup-^ V \gk{^k-i,^k) -log£;[e^®'=(«'=“i’«'=)|^fe_i] I < 0 a.e. (2.10) 

Using the inequalities logx < a; — 1 (x > 0) and 0<e® — 1 — x< (x G R), from Eq. 

(2.10), we have 

^ a„+ 4 >{n) 

limsup-— V {gk{^k-i,^k) - E[gk{^k-i,^kMk-i]} 

" fe=a„ + l 


a„+0(n^ . 

limsup-^ V -logE[e^®'=(«'=-i’«'=)|^fe_i] - E[5ffe(^fc_i,^fe)|^fe_i’ 
„ (p(n) _Li I ® 

E[(e^3kiik-iAk) _ 1 _ sgki^k-i,ik))\^k-i] 


< 


= lim sup 




k—an-\-^ 

a„+tj>{n) 

E 

fc=an + l 
an-\-(p{n) 


limsup-^ ^ 

2 " feir+i 


<-sc(7; w) < cx) a.e. 


Letting s ), 0 in Eq. (2.11), we obtain 

^ a„+0(n) 

limsup-— V [gfe(6-i,?fe) - £^(5fc(?fe-i.6)l6-i)] < 0 a.e. 

" fc=a„ + l 

Letting —7 < s < 0 in (2.9), similarly, we obtain that 

^ a„-|-0(n) 

liminf-^ [fffc(&-i,?fe) - Cfe)ICfe-i)] > 0 a.e. 

’ fc=an + l 

Eq. (2.3) follows immediately from Eqs. (2.12) and (2.13). 


( 2 . 11 ) 

( 2 . 12 ) 

(2.13) 

□ 


Let 1{.}(-) be the indicator function and Sm,n{j', w) the number of occurrences of j in the 
segment - ■ ■ , im+n-i- It is easy to see that 

m+n— 1 

k—m 

Let Sjn,n{h3‘^ ^) be the number of occurrences of the pair (i, j) in the sequence of ordered 
pairs {^mi ‘‘ 1 Then 

m+n—1 

k—m 

Corollary 1. Under the conditions of Lemma 1, let Sm,n{j]^) he defined as before. Then 

^ a„+4i{n) 

lim-p-{S'a„,0(„)(j;w) - E Pki^k-i,j)} = 0 a.e. (2.14) 

" fc=o„+l 
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Proof. Let gk{x,y) = in Lemma 1. It is easy to see that {gk{x,y),k > 1} satisfy the 

Eq. (2.2) of Lemma 1. Noticing that 

an+0(n) b 

'y ' 1J Cfe) y ^ gk{^k—lj l)pk{^k—l J 0} 

fc—ttn+l 1 

an+<p(n) b 

k—an-\-l ^—1 

ara+0(n) 

= 5'a„,0(n) (j j + l{j} (Ca„+0(n)) ~ l{i} (^o„) ~ ^ ^ Pk{^k-ljj), (2-15) 

fc^a^ + l 

Eq. (2.14) follows from Lemma 1. □ 


Corollary 2. Under the eonditions of Lemma 1, let Sm,n{h j]^) be defined as before. Then 

^ a„+4>{n) 

lim-p-{5'a„,0(„)(i,j;a;) - ^ l^,}{fk-i)Pkii,j)} = 0 a.e. (2.16) 

fe=a„ + l 

Proof. Let gk{x,y) = l{i}(x)l{j}(7/) in Lemma 1. It is easy to see that {gk{x,y),k > 1} 
satisfy the Eq. (2.2) of Lemma 1. Noticing that 

On+0(n) b 

y {5fe(Cfc —1) Cfe) y ^ gkifk — l^ l)Pk{^k—lj 0} 

k—an-\-l 1—1 

On+0(™) b 

k—an-\-l 1—1 

an+ti>{n) 

='S'o„.0(„)(*, j;w) - ^ l{i}(5fc_i)pfc(i, j), (2.17) 

k—an-\-l 

Eq. (2.16) follows from Lemma 1. □ 


Lemma 2. Let (a„)))^Q and ((/>(n))))Lg be as in Lemma 1, and h{x) be a bounded function 
defined on an interval I, and (a;„)))Eg a sequenee in 1. If 


lim 

n 


1 

fin) 


an+(i>{n) 

E 


fc = an + l 


\xk - a;| = 0 , 


and hix) is eontinuous at point x, then, 


lim 

n 


1 

fin) 


an+4>(n) 

E \hixk) - h{x)\ = 0. 

k—dn 1 


Proof. The proof of this lemma is similar to that of Lemma 2 in [12], so we omit it. □ 


Lemma 3. Let (^n))]^o be a sequence of arbitrary random variables taking values from a 
finite state-space of X. = {1,2,-•• ,b}, and let fa„,<t>(n)i<^) be defined by Eq. (1.2). Then 
/a„,<^(n)(w) are uniformly integrable. 
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Proof. To prove that /a„,0(n)(w) are uniformly integrable, it is sufficient to verify the fol¬ 
lowing two conditions (see [5], p.96) 

a) For every £ > 0, there exists (5(£) >0 such that for any A G 

P(A) < 5(e) [ /a„,0(„)(a;)(iP < £ for every n. 

J A 

b) 0(„)(u;) are bounded for all n. 

Let AgT . It is easy to see that 


/a„.0(«)(w)(iP 


^logp(ea„.0(„))rfP 

logP(2^o„,0(n)) ■ P(^ n {^o„,0(n) ^an,4>('n)}) 






< - 


E 


Xa.,. ,••• .X 


an+<i>(n) 


(f{n) 


logP(^ n {fan.<p{n) — ^an,<i>(n)}) * P(^ H {^an,<i>(n) — ^fin }) * 


(2.18) 


Replacing logP(£l n {Ca„.0(«) = a;a„,0(n)}) by log in Eq. (2.18) and noting that 


E ^ {?a„.0(„) = a:a„.0(n)}) = P(dl) = ^ 

XanP" :Xa.ri+4‘(n) Xan,---,X 

by the entropy inequality 


P(A) 

llC/>(n) + l ■ 




-^Pk ^ogpk < -y^Pfc log Qk, 






where Pk,qk>0, A: = 1,2, • • • , 5 and J2k^i Pk = J2k^i Qk, we have 


/an,</>(n) 


E 

Xan A" )2^aTi, + ^(n) 


1 P(A) 

50 (n) + l ' {^o„,0(n) = Xa^,^(n)}) 


1 / PfA) _^ 

(fM 1 6 b(")+i E P(^ Pi {^o„_ 0 („) = Xa„, 0 („)}) 


(/)(n) (p{n) J 

<(21og5-logP(A))P(pl). 


(2.19) 


Since lim2,_>o+ a;(21og6 — logx) = 0, the left hand side of Eq. (2.19) is small provided P(A) 
is small and a) holds. Letting A = 17 in Eq. (2.19), we have 

P^/a„.0(n)(w) = j /a„,0(„)(a;)(iP < 2 log 6. 

Thus b) holds and the proof of the Lemma 5 is complete. □ 
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3. The Main Results 


In this section, we will establish the strong law of large numbers for frequencies of occur¬ 
rence of states and the pairs of states for delayed sums of nonhomogeneous Markov chains 
and the generalized entropy ergodic theorem for the Markov chains. 


Theorem 1. Suppose ® nonhomogeneous Markov chain taking values from a finite 

state-space ofX. = {1, 2, • • • ,b} with the initial distribution (1-4) and the transition matrices 
(1.5). Let (an)!^o as in Lemma 1. Let (i,w) and Sa.^,,p(n)ih 

be defined as before, and /o„,<;i(n) (w) be defined by Eg. (1.6). Let P = ip(i, j))bxb be another 
transition matrix, and assume that P is irreducible. If Eg. (2.1) holds and 


then 


^ a„-|-0(n) 

\Pkii,j) -p{i,j)\=0, yi,jGX, 

n 0{n) 

’ fc=an+l 


(3.1) 


where (tti , • 
P. 


(i) \im——Sa^Mn){i;(^) = a-e- Vi € X, (3.2) 

n 0 (n) ^ ’ 

{a) = 7^rp{i,j) a.e. Vi, j € X, (3.3) 

n (pin) ^ ’ 

b b 

(iii) lim(ctj) EE 7r*p(*,j)logp(bi) a.e., (3.4) 

i=i j=i 

, TTf,) is the unigue stationary distribution determined by the transition matrix 


Remark 3. It is easy to see that if lim„p„(t, j) = p{i,j) Vi,j G X, then Eq. (3.1) holds. 
Observe that 


^ o„+0(n) 

^ E IPfc(*o')-p(*o')l < (1 + 

'' fc=a„-|-l 


an \ 1 

(j){n) Qn + 4>{n) 


an+0(n) 

E \PkiL j) - Pifi j)\- 

k=l 


If, in addition, is bounded, then Eq. (3.1) follows from the following equation 

1 ” 

lim-E bfc(b j)-P(*,j)l = 0 Vi,jGX. (3.5) 

fe=l 

But in general Eq. (3.5) may not imply (3.1). For example, let 

'1 i ' 

W2= ? f . 

- 2 2 . 

Let {fn)'r=Q b® S' nonhomogeneous Markov chain with transition matrices 

Pi, ii2’^ <n<2^ + k,k>Q, 

P 2 , otherwise. 





Let P = P 2 . It is easy to see that when 2^ < n < 2^+^, for any i,j € X 

2fc+l_i 

n ' '' - ■ ' ■ 2^ 

l + 2 + 3 + ’'- + (fc + l)l 1 (fc + 2)(A: + 1) 1 


^ n ^ 2“^ -1 


< 


2 - ^ 0 (fc- 5 > cx)). 


2fc 6 2^= 

So Eq. (3.5) holds. However, if we let a„ = 2” and (j){n) = n, then 

a„+0(n) 2"+n 

^ X bfc(Lj)- p(lj)I = - X bfc(Lj)-p(Lj)l= g, 
SO Eq. (3.1) does not hold. 


Remark 4. From Lemma 3, we know that w), j;w) and 

/o„,<^(n)(w) are all uniformly integrable, so Eqs. (3.2), (3.3) and (3.4) also hold with jCi 
convergence. 


Remark 5. The right hand side of Eq. (3.4) is actually the entropy rate of a Markov chain 
with the transition matrix P. 


Remark 6. If we define a statistic as follows: 

b b 

^=-xx 

2=1 2=1 


‘^a„,<^(ri)(*j ^) 'S'o„,0(n) (*; Ji ^) , 'S'o„,<^(n) 


log- 


(lj; 


^(n) (i; w) (i; w) 


it is easy to see from Theorem I that H is a strongly consistent estimate of entropy rate H, 
where 

b b 

H = - XX 7rjp(i, j)logp(i, j)- 

i=i j=i 

Putting a„ = 2" and <()(n) = n, under the condition of Eq. (3.1), we can use information 
from a segment of {^n)^=o to estimate the entropy rate of a nonhomogeneous Markov chain. 


Proof. Proof of (i). It is easy to see that 

an+(i>{n) a„+0(n) b 

X Pki^k-l,j)= X^T}(^fc- i)pk{i,j), Vj e X, 

k—an-\-l k—an-\-l *=1 


and 


o„+0(n) b b 

X Xif X ^n,(p{n) i‘^1 u^)p{i,j), Vj€X. 

k—an-\-l 2=1 2=1 

From (3.1), we have that 


o„+0(n) b 

liml^ X X 

)(n) 

'' '' k—an-\-l i=l 

6 - an+<?5>(n) 

it ” fcir+i 


^{i}{fk-i)[pk{i,j) - pii,j)] 
\Pk{i,j) -pii,j)\=o, Vj e X. 


(3.6) 


(3.7) 


(3.8) 
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Combining Eqs. (2.14), (3.6), (3.7) and (3.8), we have 

1 ^ 

lim-p-[-S'„,0(„)(j;a;) - ^ S'o„,0(„)(i; u;)p(i, j)] 

^ i—1 

^ an+</>(n) b 

^ IIi{i}(5fc-i)bfc(bj)-p(bj)] 

^ fc=o„ + l i=l 

=0, a.e. VjeX. (3.9) 


Multiplying the two sides of Eq. (3.9) by p{j, k), and adding them together for j = 1, 2,..., &, 
we have 

b b 

0 = lim (^(n) E .0(n)(j;w)p(j,A:) EE ^an,4>{n) (^! 3^P{^3-> ^)] 




i=i 


^ ^ 1 1 
= 1tE ^'^a„.0(„)(j;w)p(j,fc) - ^5a„,,^(„)(fc;w)] 


b b 


<S'a„,0(n) (*; W)p(i, j)p(j) *)] 

1 ^ ^ 


(3.10) 


where k) {I is a positive integer) is the /-step transition probability determined by the 

transition matrix P. By induction, for all / > 1, we have 

1 ^ 

lim-p-[5'a„_0(„)(fc;w) - E'S'a„.0(n)(bw)p^'Hi,fc)] =0, a.e., 

' 2=1 


(3.11) 


and 


1 1 ^ 1 ' \ 
lini[-^S'a„,0 („)(fc;w)E'5'a„,</>(«)(*;w)—E 7'^^^(bfc)] = 0, a.e. (3.12) 
n (p{n) 9(n) m 


It is easy to see that X]i=i 'S'o„.0(n)(b w) = </>(^); by (3.12), we have for all m > 1 

^ b ^ m 

limsup |-^S'a„,,^(„)(A:;w) -tt^I < ^ \—'^p^''\i, k) -Tr^l a.e. 
n (p(n) m 

Because P is irreducible, so 


lim — P^^\h k) = TTk, Vi € X, 

m. 7T? • ^ 


(3.13) 


(3.14) 


i=\ 


Eq. (3.2) follows from Eqs. (3.13) and (3.14). 


Proof of (ii). Observe that 

On-l-<i>(n) 

E l{d(5fc-i)p(*0') = -5'a„,0(n)(bw)p(/, j). (3.15) 
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From Eq. (3.1), we have that 


^ a„+0(n) 

lim-p- '^{t}i^k-i)[pkii,j)-pii,j)] = 0. (3.16) 

fe=o„ + l 

Combining Eqs. (2.16), (3.15) and (3.16), we have 

lim^^[S'a„,,^(„)(i, j;a;) - S'a„,0(„)(i; w)p(i, j)] 

^ a„+0(n) 

= limpw ^{i}i^k-i)[Pkii,j)-pii,j)]=0 a.e. (3.17) 

” fc=o„ + l 


Eq. (3.3) follows from Eqs. (3.2) and (3.17). 

Proof of (hi). Since Ee'by Markov inequality, 
for every £ > 0, form Eq. (2.1), we have 


^P[(/)(n) ^|log/Xo„(^o„)| > e] < 

n—1 n—1 


By Borel-Cantelli lemma, we obtain 


limlog/Xo„ (^o„) = 0 a.e. (3.19) 

n (p{n) 

Letting gk{x,y) = \ogpk{x,y) and 7 = ^ in Lemma 1, and noticing that 

i?[(logPfe(a-i, a))'e^l |a_i] 

b ^ 

= '^Pk^ {^k-l,j)^Og^ Pk{^k-l,j)Pk{^k-l,j) 
i=i 

b 

^'^PU^k-i,j)log^ Pk{^k-i,j) < 16&e"^ (3.20) 

1=1 


it follows from the Lemma 1 that 


lim 

n 


1 


a„+0(n) 

E 


fc=an + l 


logPfe(^fc-l, 


' \ 

^k) - ^Pk{^k-l,j)^OgPk{^k-l,j) > 

1=1 J 


= 0 a.e. 


(3.21) 
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Now 


an+0(n) b 


(j){n) 


'^Pk{^k-i,j)^ogpk{^k-i,j) -p{i,j) log p{i,j)\ 


k—an+lj—1 
a„+0(n) b b 


i=l j=l 


t}i^k-i)Pkii,j)logPkii,j) 

' k^art + l 7 = 1 


k—an-\-l i—1 j — 
t+4>{n) b b 


(j3{n) 


E EEi( ^}i^k-l)pii,j)logp{i,j)\ 


k—an-\-l i —1 j —1 

an-\- 4 >{n) b b 


VC 


^ E EEv i}{^k-i)p{i,j)logp{i,j) - E’^^E pihj) ^Ogp{i,j)\ 


k—an-\-l ^=1 i=l 


2=1 j = l 


b b ^ a„+0(n) 

<EE 7 pr E \Pk{hj)^ogpk{i,j) - pii,j)logp{i,j)\ 

j^l fc=an + l 


b b 


^-EE \p{i,j)\ogp{i,j)\\--^ E Iw(V-i)- 7 r*|. 

i=l j=l ' k=a„ + l 

By Lemma 2, Eq. (3.1) and the continuity of h{x) = xlogx, we have 


an+0(n) 


(3.22) 


E \Pk{i,j)^ogpk{i,j)-pii,j)logp{i,j)\=0 yi,j €X. 

’ fc=an + l 

Combining Eqs. (3.21), (3.22), (3.23) and (3.2), we have 
^ an+VC^) i> ^ 

E logPfc(V-i,V) = E^*El^(*’-C)logp(Lj) a.e. 

' fc=o„ + l i=l j=l 

From Eqs. (1.6), (3.19) and (3.24), Eq. (3.4) follows. 


T+<i>(n) 


(3.23) 


(3.24) 

□ 


Corollary 3. (see [12]). Under the eonditions of Theorem 1, if Eq. (3.5) holds, then 

(i) lim —n(i; w) = TTi a.e., and £i Vi G X, (3.25) 

n n 

(a) lira - So ^n{i, j;bo) = Tfip{i,j) a.e., and £i Vi,j G X, (3.26) 

n n 

b b 

(Hi) lim f 0 n{uj) = -y^ y^ Trip{i, j) logp{i,j) a.e., and Ci, (3.27) 

2=1 i=l 

where (tti, • • • ,7rf,) is the unique stationary distribution determined by the transition matrix 

P. 
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